Papers with natural language understanding abilities
Meaning Beyond Truth Conditions: Evaluating Discourse Level Understanding via Anaphora Accessibility (2025.acl-long)
Copied to clipboard
| Challenge: | Existing assessments of understanding at the lexical and sentence levels are limited to lexica and sentence level, but few of them target whether LLMs accurately represent and update states of natural language discourse. |
| Approach: | They propose anaphora accessibility as a diagnostic for assessing discourse understanding . they use a dataset inspired by theoretical research in dynamic semantics to evaluate human and LLM performance. |
| Outcome: | The proposed dataset shows that humans and LLMs align on some tasks and diverge on others. |
Don’t Trust ChatGPT when your Question is not in English: A Study of Multilingual Abilities and Types of LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models can perform a wide variety of language tasks when presented in English. |
| Approach: | They propose a method to evaluate the multilingual capabilities of large language models using a prompt back-translation method to find out how LLMs acquire their multilingual abilities. |
| Outcome: | The proposed method shows that large language models can transfer learned knowledge across different languages, but struggle to provide accurate results in translation-variant tasks. |